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Abstract 

Motivated by recent developments on visible actions on complex 
manifolds, we raise a question whether or not the multiplication of 
three subgroups L, G' and H surjects a Lie group G in the setting 
that G/H carries a complex structure and contains G' /G' D H as a 
totally real submanifold. 

Particularly important cases are when G/L and G/H are gener- 
alized flag varieties, and we classify pairs of Levi subgroups {L, H) 
such that LG'H = G, or equivalently, the real generalized flag variety 
G' /H n G' meets every L-orbit on the complex generalized flag vari- 
ety G/H in the setting that {G,G') = {U{n),0{n)). For such pairs 
(L, H), we introduce a herringbone stitch method to find a generalized 
Cartan decomposition for the double coset space L\G/H, for which 
there has been no general theory in the non-symmetric case. Our 
geometric results provides a unified proof of various multiplicity-free 
theorems in representation theory of general linear groups. 
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1 Introduction and statement of main results 

1.1. Our object of study is the double coset space L\G/H, where L C G D H 
are a triple of reductive Lie groups. 

In the 'symmetric case' (namely, both {G, L) and (G, H) are symmetric 
pairs), the theory of the Cartan decomposition G = LBH or its variants 
gives an explicit description of the double coset decomposition L\G/H (e.g., 
13 HI UH UH UEl IHj)- However, in the general case where one of the pairs 
{G, L) and {G, H) is non- symmetric, there is no known structure theory on 
the double cosets L\G/H even for a compact Lie group. 

Motivated by the recent works of 'visible actions' f Definition 17. ip on com- 
plex manifolds OIH] and multiplicity- free representations (e.g. the classifica- 
tion of multiplicity-free tensor product representations of GL{n), see |HIIT^). 
we have come to realize the importance of understanding the double cosets 
L\G/H in the non-symmetric case such as 

(G, L, H) = (Uin), U{n^) x [/(na) x ■ ■ ■ x U{nk), U{mr) x ■ ■ ■ x f/(mO), (1.1) 

where n = ni + ■ ■ ■ + = nii + ■ ■ ■ + m;. 

In this article, we initiate the study of the double cosets L\G/H in the 
'non-symmetric and visible case' by taking (jl.lj) as a test case, and develop 
new techniques in finding an explicit decomposition for L\G/H. 

For this, first we single out triples that give rise to visible actions. The- 
orem ^ gives a classification of the triples (L, G, H) such that G has the 
decomposition G = LG'H where G' = 0{n), or equivalently, any L-orbit on 
the complex generalized flag variety G/H ~ Smi,...,m,(C'^) (see fll.5|) ) inter- 
sects with its real form Bmi,...,mi{^'^)- The proof uses an idea of invariant 
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theory arising from quivers. The classification includes some interesting non- 
symmetric cases such as A; = 3, / = 2 and min(ni + 1, n2 + 1, n-j, + 1, mi, 7712) = 
2. Then, the L-action on G/ H (and likewise, the if-action on G / L) becomes 
visible in the sense of 0| (see Definition 17. ip . 

Second, we confine ourselves to these triples, and prove an analog of the 
Cartan decomposition 

G = LBH 

by finding an explicit subset B in G' such that generic points of B form 
a J-transversal totally real slice f Definition 17. Ij) for the L-action on G/H 
of minimal dimension (see Theorem |B|) . The novelty of our method in the 
non-symmetric case is an idea of 'herringbone stitch' (see Section 

1.2. To explain the perspectives of our generalization of the Cartan decom- 
position, let us recall briefly classic results on the double cosets L\G/H in 
the symmetric case. A prototype is a theorem due to H. Weyl: let i^' be a 
connected compact Lie group, and T a maximal toral subgroup. Then, 

any element of K is conjugate to an element of T . (1-2) 

We set G := KyiK, A:= {(t, t'^) : t e T} and identify K with the subgroup 
diag(A') := {{k, k) : k & K} of G. Then, the statement ()1.2p is equivalent to 
the double coset decomposition: 

G = KAK. (1.3) 

In the above case, (G, K) = {KxK, dia.g{K)) forms a compact symmetric 
pair. More generally, the decomposition (jl.3p still holds for a Riemannian 
symmetric pair {G,K) by taking A ~ {G/K: compact type) or A ~ M*^ 
{G/K: non-compact type) where k = Ta.nkG/K. Such a decomposition is 
known as the Cartan decomposition for symmetric spaces. 

A further generalization of the Cartan decomposition has been devel- 
oped over the decades under the hypothesis that both (G, L) and (G, H) 
are symmetric pairs. For example, Hoogenboom |3] gave an analog of the 
Cartan decomposition for L\G/H = {U{1) x U {m))\U {n) / {U {p) x U{q)) 
{I + m = p + q = n) hy finding a toral subgroup T'^ as its representatives, 
where k = min(Z, m,p, q). This result is generalized by Matsuki showing 
that there exists a toral subgroup B in G such that 

G = LBH (1.4) 
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if G is compact (see Fact 12.11) . Analogous decomposition also holds in the 
case where G is a non-compact reductive Lie group and L is its maximal com- 
pact subgroup, by taking a non-compact abelian subgroup B of dimension 
rankiuG/if (see Flensted- Jensen 

1.3. Before explaining a new direction of study in the non-symmetric case, 
we pin down some remarkable aspects on the Cartan decomposition in the 
symmetric case from algebraic, geometric, and analytic viewpoints. 

Algebraically, finding nice representatives of the double coset is relevant 
to the reduction theory, or the theory of normal forms. For example, the 
Cartan decomposition ()1.3|) for {G^K) = {GL{n,'R),0{n)) corresponds to 
the diagonalization of symmetric matrices by orthogonal transformations. 
The case G = G' x G',L = H = diag(G") with G' = GL{n, C) is equivalent 
to the theory of Jordan normal forms. 

Geometrically, ()1.4|) means that every L-orbit on the (pseudo-)Riemannian 
symmetric space G/H meets the fiat totally geodesic submanifold B/B (IH. 
The decomposition ()1.4|) is also used in the construction of a G-equivariant 
compactification of the symmetric space G/H (see j2] for a survey on various 
compactifications) . 

Analytically, the Cartan decomposition is particularly important in the 
analysis of asymptotic behavior of global solutions to G-invariant differential 
equations on the symmetric space G/H (e.g. [Sllllini)- 

1.4. Now, we consider the non-symmetric case L (Z G D H. Unlike the 
symmetric case, we cannot expect the existence of an abelian subgroup B 
such that LBH contains an interior point of G in general, as is easily observed 
by the argument of dimensions (e.g. |6, Introduction]). Instead, we raise here 
the following question: 

Question 1.1. Does there exist a 'nice' subgroup G' such that LG'H contains 
an open subset of G? 

For a compact G, one may strengthen Question 11.11 as follows: 

Question 11.11 . Does there exist a 'nice' subgroup G' such that G = LG'H? 

The decomposition G = LG'H means that the double coset space L\G/H 
can be controlled by a subgroup G'. 

What is a 'nice' subgroup G'? In contrast to the previous case that G/H 
carries a G-invariant Riemannian structure and that the abelian subgroup 
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G' = B (see (II. 4j) ) gives a flat totally geodesic submanifold of G/H^ we 
are interested in the case that G/ H carries a G- invariant complex structure 
and that the subgroup G' gives a totally real submanifold G' /G' fl if of 
G/H. In the latter case, the L-action on G/H is said to be previsible (0 
Definition 3.1.1]) if LG'H contains an open subset of G. 

1.5. Let us state our main results. Suppose we are in the setting ()1.1|) and 
consider Question ll.il for 

G' ■= 0{n). 

In this setting, we shall give a necessary and sufficient condition for the 
multiplication map L x G" x i7 ^ G to be surjective. 

In order to clarify its geometric meaning, let Smi,...,m,(C") denote the 
complex (generalized) flag variety: 

{(V^i, . . . , V^) : {0} = C 1^1 C 1^2 C ■ ■ ■ C C = C", 

<lmvVi = mi^ Vnii (1 < i < /)}. (1.5) 

Likewise, the real (generalized) flag variety Bm-^,...,mliW) is deflned and be- 
comes a totally real submanifold of i3mi,...,m;(C"). 

Theorem A. Let k,l > 2 and n = ni + ■ ■ ■ + nk = mi + ■ ■ ■ + be partitions 
of n by positive integers. Let 

(G, L, H) = {U{n), U{ni) x U{n2) x ■ ■ ■ x U{nk), U{mi) x ■ ■ ■ x U{mi)). 

We set N := min(ni, . . . , n^) and M := min(mi, . . . , mi). Then the following 
five conditions are equivalent: 

i) G = LG'H. Here, G' ■.= 0{n). 

ii) i3m,,...,m,(]R") X i3„,,...,„,(]R'') meets every G-orbit on Bmi,...,mi{C) x 
'Bni,...,nfc(C"') by the diagonal action. 

ii)' Bmi,...,mi{^^) rneets every L- orbit on Bmi,...,mi{^^) ■ 

ii) " Bni,...,nk(M"') rneets every H- orbit on Bni,...,n,,{^"') . 

iii) One of the following conditions holds: 
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0) k = 2, 

1) A; = 3, = 1, 
II) A; = 3, iV > 2, 
III) 

I') k = 2, 

IV) k = 2, N = 2, 

III') k = 2, N = 1. 



2, 
2, 

2, M = 2, 

2, M = l, 

3, M = l, 
3, M>2, 



Remark 1.5.1. Both {G,L) and {G,H) are symmetric pairs if and only if 
(fc, I) = (2, 2), namely, (G, L, H) is in Case 0. 

Remark 1.5.2. The condition (ii) implies that the L-action on G/H is previsi- 
ble (see Definition l7.1|) . We shall see in Section[7|that this action is (strongly) 
visible, too (see also P, Corollary 17]). 

Remark 1.5.3. A holomorphic action of a complex reductive group on a com- 
plex manifold D is called spherical if its Borel subgroup has an open orbit 
on D. We shall see in Section |S1 that the condition (ii) in Theorem |X] is 
equivalent to: 

iv) i3mi,...,mi(C") X Bni,...,nk{C"') IS a sphcrical variety of Gc '■= GL{n,C). 

See Littelmann [T^j for the statement (iv) in the case k = I = 2, namely. 
Case in (iii). 

Remark 1.5.4. An isometric action of a compact Lie group L on a Riemannian 
manifold is called polar if there exists a submanifold that meets every L-orbit 
orthogonally. Among Cases 0~III' in Theorem IXl the L-action on G/H is 
polar if and only if {G,L,H) is in Case (see [T]). 

1.6. Suppose one of (therefore, all of) the equivalent conditions in Theo- 
rem [X] is satisfied. As a finer structural result of the double coset decompo- 
sition L\G/H, we shall construct a fairly simple subset B of G' = 0{n) such 
that the multiplication map LxBxH^G is still surjective, according to 
Cases ~ III of Theorem ^ We omit Cases I' ~ III' below because these 
are essentially the same with Cases I ~ III. For Case I below, we may and 
do assume rii = 1 without loss of generalities. 

Theorem B (generalized Cartan decomposition). Let 

(G, L, H) = {U{n), U{n{) x ■ ■ ■ x [/(n^), U{p) x U{q)). 
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Then, there exists B C 0{n) such that G = LBH , where B is of the following 
form: 

''jmm(ni,n2,p,<?) (Q^gg g), 

'jpmin{p,(3,n2,n3) . ipmin{p,g,n2+l,n3+l) (^Qagg X) 

T2.T-T2 (Case II), 

T- ■ ■ ■ -T (Case III). 

fc-i 

Here, ■ means a subset of the form {xy E G : x E T", y G T^} for some 
toral subgroups T° and T'^. 

We note that B is no longer a subgroup of G in Cases I, II and III. 

1.7. This article is organized as follows. First, we give a proof of Theorem iBl 
(generalized Cartan decomposition G = LBH) by constructing explicitly the 
subset B of 0{n). This is done in Theorems 12.21 (Case 0), Kill (Case I), 14.11 
(Case II), and 15.11 (Case III), respectively by using an idea of herringbone 
stitch. This also gives a proof of the implication (iii) =^ (i) in Theorem 1X1 The 
remaining implications of Theorem^are proved in SectionlHl An application 
to representation theory is discussed in Section |S| 

Theorem 1X1 was announced and used in [HI Theorem 3.1] and [HI The- 
orem 16] and its proof was postponed until this article. Theorem El was 
presented in the Oberwolfach workshop on "Finite and Infinite Dimensional 
Complex Geometry and Representation Theory" , organized by A. T. Huck- 
leberry, K.-H. Neeb, and J. A. Wolf, February 2004. The author thanks the 
organizers for a wonderful and stimulating atmosphere of the workshop. 



2 Symmetric case 

This section reviews a well-known fact on the Cartan decomposition for the 
symmetric case. The results here will be used in the non-symmetric case 
(Sections El in andE)) as a 'stitch' (see Diagram 13. H for example). Theorem 
12.21 corresponds to Theorem El in Case 0, which is proved here. 

First, we recall from ^ Theorem 6.10], fSl Theorem 1] the following: 

Fact 2.1. Let G be a connected, compact Lie group with Lie algebra g. Sup- 
pose that T and 9 are two involutive automorphisms of G, and that L and H 
are open subgroups of G'^ and G^ , respectively. We take a maximal abelian 



7 



subspace b in q ^ := {X G g : tX = 9X = —X}, and write B for the 
connected ahelian subgroup with Lie algebra b. Then G = LBH . 

Now, let us consider the setting: 

(G, L, H) = {U{n), f/(ni) x U{n^), U{p) x U{q)), (2.1) 

where n = ni+n2 = p+q. Then, both {G, L) and (G, H) are symmetric pairs. 
In fact, if we set Ip^q := diag(l, . . . , 1, —1, . . . , —1) and define an involution 
6 by 6{g) := Ip^qglp q~^, then H = G^. Likewise, L = G'^ if we set T{g) : = 

We set 

Z := min(?T.i, ?T.2,p, g) (2.2) 
and define an abelian subspace: 

I 

b := I^(-£'i,n+l-i — En+l-i,i)- 

i=l 

Then, b is a maximal abelian subspace in 0^^'^^, and B := exp(b) is a toral 
subgroup of 0(n). Now, applying Fact 12. H we obtain: 

Theorem 2.2. G = LBH. 

3 Non-symmetric case 1: min(ni, n2, 7^3) = 1 

In Sections El IH and we give an explicit decomposition formula for the 
double coset space L\G/H in Cases I, II, and III of Theorem|Bl respectively, 
and complete the proof of Theorem ^ and therefore that of the implication 
(iii) =^ (ii) of Theorem 1X1 

The distinguishing feature of these three sections is that we are dealing 
with the non-symmetric pair {G,L), for which there is no known general 
theory on the double coset decomposition L\G/H of a compact Lie group 
G. We shall introduce a method of herringbone stitch (see Diagram 13. ip 
consisting of symmetric triples G2i+i C G2i D Hi and Lj C G2j+i ^ ^24+2 
(i = 0,1,2,...) such that the iteration of the double coset decomposition 
of G2i+i\G2i/ Hi and Li\G2i+i/G2i+2 keeps on toward a finer structure of 
L\G/H = Lo\Go/Ho. 
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This section treats the most interesting case for Theorem ^ namely, 

(G, L, H) = {U{n), U{ni) x U{n2) x U{ns), U{p) x U{q)) (3.1) 

where min(ni, n2, ^3) = 1. Theorem 13.11 below corresponds to Case I of 
Theorem El 

Without loss of generality, we may and do assume ni = 1. Thus, L = 
U{1) X U{n2) X [/(ng) (na + ng = n - 1). We set 

/ := min(2p,2g,2n2 + l,2n3 + 1). (3.2) 

A simple computation shows [|] = mm{p, q, n2, n^} and [^] = min(p, q, n2+ 
l,n3 + 1). We define two abehan subspaces: 

[^] 

b' := ^ R{Ei^n+i-i — En+i-i,i), 
1=1 

[|] 

b" I^(-£'j+l,n+l-j ~ En+l-i^i+l). 

i=l 

Then := exp b' and -B" := exp b" are toral subgroups of dimension [^] 
and [|], respectively. We define a subset of 0{n) by 

5 := B"B'. (3.3) 

For the sake of simplicity, we shall write also B = T^^l ■ We note 

that i? is a compact manifold of dimension / = [|] + [^] because B is 
diffeomorphic to the homogeneous space {B' x B") / (B' fl B") and B' fl B" is 
a finite subgroup. 

We are ready to describe the double coset decomposition for L\G/H in 
the case (I3.1|l . 

Theorem 3.1 (generalized Cartan decomposition). G = LBH, where 

'jpmin{p,g,n2,ra3) _ ipmin(p,q,n2+l,n3+l) 

Proof. First, for i > 1, we define one dimensional toral subgroups by 

Bi := expR{Ei^n+i-i — -£^n+i-j,i), 

Ci := exp]R(£'j+i^„+i_j — En+i-i^i+i). 
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Then, Bi, B2,- ■ ■ , and B^i+i^ commute with each other, and we have 
B' = B1B2 ■ ■ ■ -S[i±i] = ■ • • B2B1. 

Likewise, B" = CiC2---C^i]. 

For m > 1, we define an embedding of a one dimensional torus into G by 
Lm '■ ^ ^ G, a diag(a, . . . , a, 1, . . . , 1, a, . . . , a). 

[ "1+1 ] n—m [^] 

We write Tm for its image, and define a subgroup Gm of G by 

Gm := Tm y. U{n - m). 

Here, we regard U{n — m) as a subgroup of G by identifying with {/[m+i]} x 
U{n — m) X {/[Hi]}. {Ira stands for the unit matrix of degree m.) It is 
convenient to set Tq = {e} and Go = G. Then, we have a decreasing sequence 
of subgroups: 

G = Gq 3 G\ 3 G2 Z) ■ ■ • . 

The point of our definition of Gm is that 

(j2i commutes with all of ■ ■ ■ , Bi, Ci, ■ ■ ■ , Gi-i. 
G2i+i commutes with all of ■ ■ ■ , -Bj, Ci, ■ ■ ■ , Cj. 

Next, for i > 0, we define the following subgroups: 

Hi■.= HnG2^ ^T2iXU{p-i)xU{q-i), 
Li:= Ln Gai+i ^ ^2^+1 x U{n2 -i) x U (723 - i). 

The following obvious properties play a crucial role in the inductive step 
below. 

H = Ho D Hi D H2 D ■ ■ ■ ; Hi commutes with Bi, . . . , Bi, (3.4) 
L = Lq D D L2 D ■ ■ ■ ; Li commutes with Ci, . . . , Cj. (3.5) 

With these preparations, let us proceed the proof of Theorem 13. II along a 
herringbone stitch consisting of triples {G2i+i,G2i, Hi) and (1^1,^21+1,^21+2) 
(^ = 0,1,2,---): 
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Lq Li Z/2 L-} 

o o o o 

Gi Gs G5 Gy 

^2 *-^4 *-^6 *-^8 

G C G C C 

Hq Hi H2 H4 



Diagram 3.1 

We claim that each triple has the following decomposition formula: 

G2i = 6*2^+1 -Sj+i-f^j, (3-6) 
^2?;+! = LiGi+iG2i+2- (3-7) 

To see this, we first take away the trivial factor from G2i and 6*21+15 respec- 
tively. Then, the following bijections hold for i = 0,1, . . . , 

G2^+l\G2^/H, ~ (f/(l) X U{n - 22 - l))\U{n - 2t)/{U{p - t) x U{q - z)), 

L,\G2^+l/G2^+2 ^ (f/^ - >< - " 2^ " - 2t - 2) X U{1)). 

Since the right-hand side is the double coset space by symmetric sub- 
groups, we can apply Theorem 12.21 Thus, (jH.fjp and ()3.7|1 have been proved. 

By using ()3.(ip and ()3.7j) iteratively, together with the commutating prop- 
erties f|3.4|) and ()3.5|) . we obtain 

G = Go ~ G\B\Hq 

= {LqGiG2)BiH 

= LGi{G'iB2Hi)BiH 

= LG\G-:^B2B\H 

= LGi- ■ ■GiG2iBi- ■ ■ BiH (3.8) 

= LGi ■ ■ ■ GiG2i+iBi^i ■ ■ ■ BiH. (3.9) 



If min(p, q) < min(n2, n^), then this equation terminates with G2i = Hi when 



i reaches min(p, g). Then, i = [i] = [^], and we have G = LB" B' HiH 
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LBH from (j3.8|) . If min(p, g) > min(n2,n3), then this equation terminates 
with G2i+i = Li when i reaches min(n2,n3). Then, i = [|] and i + 1 = [^], 
and we have G = LLiB" B' H = LBH from ()3.9|) . Hence, we have completed 
the proof of Theorem. □ 



4 Non- symmetric case 2: min(ni, n2, ^3) > 2 

In this section we study the double coset space L\G/H in Case II of Theo- 
rem El that is, the non-symmetric case: 

{G,L,H) = {U{n),U{m) X UM x U{ns),U{p) x U{q)), 

where n = ni + n2 + = P + Q, min(ni, n2, ^3) > 2 and min(p, g) = 2. 
Without loss of generality, we may and do assume p = 2. 
We define three abelian subspaces: 

Oi := M(Ei_„ — En,i) + ]R(-E2,n-i — -En-1,2), 
a' := ]R(i?n^+i^n_i — En-i^m+i) , 

and correspondingly three toral subgroups by := exp ai. A' := exp a', and 
A2 := exp a2- We then set 

B := A2A'Ai. (4.1) 

Note that i? is a five dimensional subset of 0{n), but is no more a subgroup. 

Here is a generalized Cartan decomposition for L\G/H in the non-symmetric 
setting min(r;,i, n2, ^3) > 2 and min(p, q) = 2: 

Theorem 4.1. G = LBH. 

Proof. The proof again uses herringbone 'stitch', of which each stitch is a 
special case of Theorem 13.11 or Theorem 12.21 respectively. 
Step 1. We define a subgroup Gi of G by 

Gi := U{ni) X U{n2 + n3). 

Since both (G, H) and (G, Gi) are symmetric pairs, we have 

G = GiA^H (4.2) 
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by Theorem 12.21 We observe from (|2.2j) that Ai is of dimension min(2,ri — 
2,ni,n2 + 723) = 2. 

Step 2. We define a subgroup Hi of G by 

Hi := ZhhcMi) - A(T2) X U{m - 2) x U{n2 + m - 2). 

Here, A(T^) := {diag(a, 6, 1, ■ ■ ■ , 1, 6, a) : a,b E T}. Then, taking away the 
first factor inclusion U{ni), we have 

L\Gi/Hi ~ {U{n2) X f/(r23))\f/(n2 + n3)/(f/(n2 + ^3 - 2) x f/(l) x [/(I)). 

Applying Theorem 13. II to the right-hand side, we obtain 

Gi = LA2A'Hi. (4.3) 

We note that A2A' is of dimension min(2n2, 2n3, 2(n2 + 77.3 — 2) + 1, 3) = 3, 
as explained in Sectional 

Combining (j4.2j) and (j4.3p . we obtain 

G = GiAiH 

= {LA2A'Hi)AiH 
= LA2A'AiH 
= LBH 

because Hi commutes with Ai. Thus, we have shown Theorem 14.11 □ 

5 Non- symmetric case 3: min(2?, q) = 1 

This section treats the double coset space L\G/H in Case III of Theorem IHj 
that is, the non-symmetric case: 

(G, L, H) = {U{n), U{ni) x ■ ■ ■ x UM, U{1) x U{n - 1)) 

for an arbitrary partition n = rii + • • ■ + n^. In this case, although the 
pair {G, L) is non-symmetric, the symmetric pair [G, H) gives a very simple 
homogeneous space, namely, G/H ~ P"'~^C. Thus, it is much easier to find 
an explicit decomposition G = LBH than the previous cases in Sections El 
andm 
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For 1 < i < A; — 1, we set 

Hi :— —Ei^n]_+-+ni+l + -E'ni+-+ni+l,l 

Si := exp(Ri/i) (-T), 
and define a (/c — l)-dimensional subset B 'm. G' = 0{n) by 
B :— Bi - ■■ -Bfe-i- 

We note that B is not a group if A; > 3. The subset B is contained in the 
subgroup 0{k) of 0{n) in an obvious sense. 
Then, we have 

Theorem 5.1. G = LBH, where B = T- • • • -T (c 0{n)). 

k-l 

Proof. We shall work on the L-action on G/H ~ P"~^C. First, we observe 
that the map 

U{ni) X M ^ C^', {hi, ai) ^ hi\ai, 0, . . . , 0) 

is surjective for each i [1 <i <k). Therefore, the following map 

[/(m) X • • • X U{nk) X P'^-^R ^ P"-^C, (5.1) 
((/ii,...,/ifc),[ai : ••• : Ofc]) i-^ [/ii *(ai, 0, . . . , 0) : ••• : /i^ *(afc, 0, . . . , 0)] 

is also surjective. 

Next, an elementary matrix computation shows 

eMdiHi) ■ ■ ■ cxp{9k-iHk-i) tl, 0, . . . , 0) 



where 







ai, . . . , 0, a2, . 


. ,0, . . . ,afc,0, . . . ,0), 


ai \ 




/ cos 9i cos 02 


• • • cos 0k-2 cos 0k-l^ 


a2 




sin 9i cos 02 


■■■ cos6'fc_2 cos6'fe_i 


as 




sin ^2 


••• C0S^fc_2C0S^fc_i 








sin 9k-2 cos ^fc-i 






V 


sin 9k-i J 



e M*=\{0}. 



Hence, S • [1 : : • • • : 0] = P^^-^R. 

Combining with the surjective map L x P*^~-'-R — > G/H, we have shown 
that the multiplication map LxBxH^G is surjective. □ 
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Now, Theorems 12.21 13.H 14.11 and 15.11 show that we have completed the 
proof of Theorem ^ 

6 Proof of Theorem El 

This section gives a proof of Theorem ^ 

Suppose we are in the setting of Theorem El The equivalence (i) 4^ 
(ii) (likewise, (i) -v^ (ii)' and (i) -vv- (ii)") is clear from the following natural 
identifications: 

G'/G' n if ~ i3^„...,„,(M"), G/H Bm,,...,mXCn- 

Further, Theorem iBl shows (iii) =^ (i). 

Thus, the rest of this section is devoted to the proof of the implication 
(i) (iii). 

We begin with a question about when the multiplication map 

L X G' X H ^ G, {l,g',h) ^ Ig'h (6.1) 

is surjective, in the general setting that G = U{n), G' = 0{n), and ii is a 
Levi subgroup G. Our key machinery to find a necessary condition for the 
surjectivity of ()6.H) is Lemma 16.31 Let us explain briefly the ideas of our 
strategy that manages the three non- commutative subgroups L, G' and H: 

L ■ ■ ■ finding L-invariants (invariant theory), 

G' ■ ■ ■ using the geometric property [G' gives real points in G/H), 

H ■ ■ ■ realizing H as the isotropy subgroup of the G-action. 

For this, we take J G M(n, M) and consider the adjoint orbit: 

G/Gj {Adig)J : g e G} C M{nX), 

where Ad{g)J := gJg^^ and 

Gj:={geG:gJ=Jg}. 

Later, we shall choose J such that H is conjugate to Gj by an element of G' . 
Here, we note that the surjectivity of the map ()6.1|) remains unchanged if we 
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replace H with aHa ^ and L with hHh ^ {a,b E G'). Then, the following 
observation: 

G'Gj/Gj ~ {Ad{g)J -.geG'jc M{n,R) 

will be used in Lemma f6. II ('management' of G' ~ 0{n)), while an invariant 
theory will be used in Lemma f 'management ' of L ~ U{ni) x ■ ■ • x U{nk)). 



Lemma 6.1. Let J G M(n,]R), G' = 0{n), and L a subgroup of G = U{n). 
If there exists g & G such that 

Ad{L){Ad{g)J) n M{n, R) = 0, (6.2) 

then G ^ LG'Gj. 

Proof. First we observe that 

Ad{G'Gj)J = Ad{G')J C M{n,R). 

Then, the condition implies Ad{Lg)J n Ad{G'Gj)J = 0, whence Lg fl 
G'Gj = 0. Therefore, g i LG'Gj. □ 

Next, we fix a partition n = rti + ■ ■ ■ + n^, and find a sufficient condition 
for ()6.2|) in the setting: 



L = U{ni) X • • ■ X U{nk). 

For this, we fix /> 2 and take a loop io ii 
non- negative integers 1, . . . , such that 

io = ih ia-i ^ia {a = 1,2, . 
Now, let us introduce a non-linear map: 



I). 



ii consisting of 
(6.3) 



A 



as follows: Let P G M(n, C), and we write P as {Pij)i<i,j<k in block matrix 
form such that the (i, j)-block Pij G M{ni, rif, C). We set Pjj G M{ni, Uj] C) 
by 

(«<j), 



P 



U > J)- 



Then, Ai^^...i^{P) is defined by 



(6.4) 
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Lemma 6.2. If there exists a loop ii ^ ■ ■ ■ ^ ii {= io) such that at least 
one of the coefficients of the characteristic polynomial det(A/„.^ — Ai^y..i^{P)) 
is not real, then 

Ad{L)PnM{n,R) =0. 

Later, we shall take P to be Ad{g)J and apply this lemma to the following 
loops: 



1) 1 ^ 2 ^ 3 ^ 1, 



2) 1^3^2^4^1, 



^1231 — -P12-P23-P13 



^13241 — -^13-^2*3-^24-^14 



(6.5) 



(6.6) 



3) 1^4^2^4^3^4^1, 

A p p* p p* p p* 

^1424341 — 14-^ 24-* 24-^ 34-* 34-1 14 



(6.7) 



Proof. For a block diagonal matrix / 



V 



G L, the transform 



P Ad(/)P induces that of the (i, j)-block matrix: 

Pij^kPijlf {l<i,J<k). 

Then P*- is transformed as P*^ i-> {kPijlJ^)* = ljP*jl~^. Hence, Pij is trans- 
formed as 

Pij^kPijlj^ (l<z,j<fc), 
and then AiQ...i^ (P) is transformed as 

Therefore, the characteristic polynomial of Ai^...^ (P) is invariant under the 
transformation P ^ Ad{l)P. In particular, if Ad(L)P n M{n,R) ^ 0, then 
det(A/„.^ — Aj„...jj(P)) G ]R[A]. By contraposition. Lemma follows. □ 

Here is a key machinery to show the implication (i) ^ (iii) in Theorem 1X1 
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Lemma 6.3. Let n = ni + - ■ ■+nfc be a partition, and L = U{ni) x • ■ ■ x U{nk) 
be the natural subgroup ofG = U{n). Suppose J is of a block diagonal matrix: 



J :-- 



\ 



\ 



J2 



J, 



e M{n,R), 



(6.8) 



where Ji G M(r2j,]R) {1 < i < k). If there exist a skew Hermitian matrix 
X G \x{n) and a loop iq —>■ ii —>■■■■—>■ ii (= zq) {see ()6.3p ) such that 

det(AJ„,^-A,„...,([X,J]))^M[A], 

then the multiplication map L x G' x Gj —>■ G is not surjective. Here, we 
recall G' = 0{n). 

Proof. We set P{e) := Ad(exp(£:X)) J. In view of Lemmas 16. II and 16.21 it is 
sufficient to show 

det(A4^^-A,,...,(P(e)))^M[A] 

for some e > 0. We set Q := [X, J]. The matrix P{e) depends real analyti- 
cally on e, and we have 

P{e) = J + eQ + 0{e^), 

as e tends to 0. In particular, the (z, j)-block matrix Pij{s) (g M{ni, nj; C)) 
satisfies 

Pijie) = eQij + 0{€^) (s^O) 
for i ^ j. Then, we have 

det(A4,^ -A,,...,,(P(£))) 
= det(AJ„,^ - e'Qion " " " Qk-rk + 0{e'+^)) 



r=0 

where hr{e) (0 < r < nj^) are real analytic functions of e such that 
det(A4,^ - A,...,XQ)) = 5^ A"»o-'-/,,(o). 



(6.9) 



r=0 
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From our assumption, this polynomial is not of real coefficients, namely, 
there exists r such that hr{0) ^ M. It follows from (j6.9|) that det(A/„.^ — 
Aig...i^{P{e))) ^ ]R[A] for any sufficiently small e > 0. Hence, we have shown 
Lemma. □ 

For the applications of Lemma 16.31 below, we shall take a specific choice 
of a skew Hermitian matrix X G u{n). According to the partition n = 
rii + ■ ■ ■ + Uk, we write X = (-^ij)i<ij<fc as a block form. We note that the 
(i, j) block of Q = [X, J] is given by 

Qij = XijJj — JiXij . (6.10) 

We also note that if J G M{n, M) is a diagonal matrix whose entries consist 
of non-negative integers 1,2,...,/ such that 

#{1 < a < n : Jaa = i} = rrii {1 <i <l), 

then Gj is conjugate to U{mi) x U{m2) x ■ ■■ x U{mi) by an element of 
G' = 0{n). Since the surjectivity of (jb.lj) remains unchanged if we take the 
conjugation of H by G', we can apply Lemma to study the surjectivity 
of (jnH) in the setting jHH). 

Now, we apply Lemma 16.31 to show the following four propositions: 

Proposition 6.4. Let n = ni + n2 + = mi + m2 + be partitions of n 
by positive integers. We define (natural) subgroups L and H of G = U{n) by 

L := f/(ni) X U{n2) x [/(ng), 
H := [/(mi) X U{m2) x U{m^), 

and G' := 0{n). Then, G ^ LG'H. 

In the following three propositions, we set 

{G,L,H) = {U{n),U{ni) X ■■■ xU{nk),U{p) xU{q)) and G' = 0{n) 

where n = ni + n2 + ■ ■ ■ + = p + q. 

Proposition 6.5. G ^ LG'H if 

min(p,g)>3. A; = 3 and min(ni, n2, ns) > 2. (6-11) 
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Proposition 6.6. G ^ LG'H if 

min(p, g) = 2 and k > 4. (6.12) 
Proposition 6.7. G ^ LG'H if 

min(p, g) > 3 and k > A. (6.13) 
Proof of Proposition \6.4\ We consider the partition n = ui + n2 + and 



the loop 1 ^ 2 ^ 3 ^ 1. We take J = diag(ji, . . . , j„) G M{n,R) to be a 
diagonal matrix with the following two properties: 

jl — 1) jVii = 2, jVii+n2+l — 3, 

^{k : jk = i} = rrii (1 < i < 3). 

Then, the isotropy subgroup Gj is conjugate to H by an element of 0{n). 
We fix z G C and define a skew Hermitian matrix X = (-^ij)i<jj<3 G u(r2) 
as follows: 



Then it follows from ()6.10p that Q = [X, J] has the following block entries: 



and therefore we have 



^1231 (<5) — Q12Q23Q13 — ■ 



Thus, det(A/„, - Ai23i{Q)) = A"^ - 2zX''^-\ This does not have real 
coefficients if we take z ^ M. Hence, Proposition follows from Lemma IfT!?! □ 

Proof of Proposition ^.!^ We shall apply Lemma IfT!?! with A; = 3 and the loop 
1— >2— >3— >1. In light of the assumption ()6.11|) . an elementary considera- 
tion shows that there exist positive integers pj, (1 < "i < 3) satisfying the 
following equations: 

n =^1 + ^2 + ^3 

P = Pi + P2 + P3 
+ + + + 

q = qi + q2 + q3 
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We set 



J^■.-- 



V 



GM(ni,M) (l<i<3). 



0/ 



Then Gj ~ f/(pi + p2 + Vz) ^ U{qi + q2 + qs) = H, where J is defined as in 
Now, let us take a specific choice of X G u{n) as follows: for z E C, 



Xu := I 



X23 ■ 



This is possible because rii > 2 {i = 1,2,3). Then the (i,j)-block of Q 
[X, J] amounts to 



Q 



12 







Q23 = I I , Q 



-1' 



13 







Associated to the loop 1^2— >3— i^l, we have 



^1231 (Q) = Q12Q23Q 



13 







Hence, det(AJ„, - AusiiQ)) = X''^ + X^'^'^z ^ R[X] if we take z ^ R. Now 



Proposition 16.51 follows from Lemma 1^ 



□ 



Proof of Proposition \f).fA With the notation as in (jfi.Sj) . we define J by set- 
ting /c = 4 and 



Ji 



1 







J2 



1 







J3 = 0, J4 = 0. 
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Then the isotropy subgroup Gj is conjugate to ~ U{2) x U{n — 2) by an 
element of G' = 0{n). We consider the loop 1— s>3— s>2— >4^1. Then it 
follows from ()6.10|) that 



A 



13241 



(Q) 



Q13Q23Q24Q14 

( — -^1-^13) (~-^2-^23)*(~<^2-^24)( — -^1-^14) 

/{d,b){d,c) ■■■ 0\ 







V 







where a,b E C"^ denote the first row vectors of X13 , X23 and c,dE C"* 
denote the first row vectors of X14, X24, respectively. Since rii, 77-2, and 
714 are positive integers, we can find X G u{n) such that (a, c) ^ M. 
Then 



det(AJ„, - Ai324i(g)) = A"^-i(A - {d,b){d,c)) 
Thus Proposition 16.61 follows from Lemma 16.31 



M[A]. 



□ 



Proof of Proposition 6. ? , Without loss of generality, we may and do assume 
^ ^ ni < n2 < ■ ■ ■ < Uk- We give a proof according to the following three 
cases: 

Case 1) A; > 5. 

Case 2) k = A and ns > 1. 

Case 3) k = 4 and = 1. 

Case 1) Suppose k > 5. Then, + ■ ■ ■ + > I. In fact, if it were 
not the case, we would have k = 5 and = 1, which would imply ni = 
n2 = ■ ■ ■ = = 1 and n = 5. But, this contradicts to the assumption 
?T, = p + g>3 + 3 = 6. Now, we apply Proposition 16.51 to L' = U{ni + ^2) x 
U (723 + n^) X U (77,5 + ■ ■ ■ + rik) {~D L), and conclude 

G ^ L'G'H D LG'H. 

Case 2) Suppose k = 4 and > 1. Then, 2 < ^3 < 724 and 2 < ni + n2. 
Then apply Proposition 16.51 to L' = U{ni + 722) x ?7(n3) x U{n4) and we 
conclude that 

G 2 i^'G'i/ D LG'i/. 
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Case 3) Suppose k = 4 and 113 = 1. Then, 

k = 4 and rii = n2 = = 1. 
In the setting ()6.8|) . we define J by setting 

/I 



(6.14) 



J^ = J2 = J3 := (1) e M(l, M) and J4 := 



p-3 



V 



0/ 



Then, Gj ~ f/(p) x U{q) = H. 

Given Y = {vi, . . . ,Vp) G M{q,p; C) (f 1, . . . , fp G C''), we define X G u(?2) 

by 

/O -F* 

^ - 

/ Y*\ 

Then, Q := [X, "^l = ( y g j • Associated to the partition n = rii + n2 + 
ng + ^4 (= 1 + 1 + 1 + (n - 3)) and the loop l-*4->2-^4-^3->4^1, 

^1424341 (Q) = <5l4<524Q24Q34<534Ql4 

?^iy \V2j \V2j yVsJ \V3j \Vl 
= (w2,i;i)(w3,^^2)(^^l,W3), (6.15) 

where denotes the zero vector in C^^'^. Since p > 3 and q > 3 (> 2), we 
can take 

\ 0/ 

so that {v2,Vi){v3,V2){vi,Vs) = 1 + z. Thus, 

det(A/i - Ai42434l([X, J])) = A - 1 - 2 ^ M[A] 
if 2; ^ M. Hence Proposition follows from Lemma [6.31 □ 
Hence, the proof of Theorem El is now completed. 
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7 Visible actions on generalized flag varieties 

Suppose a Lie group L acts holomorphically on a connected complex manifold 
D with complex structure J. 

Definition 7.1 (see [9, Definitions 2.3, 3.1.1 and 3.3.1]). The action is pre- 
visible if there exists a totally real submanifold 5* such that 

D' := L ■ S is open in D. (7-1) 

The previsible action is visible if 

JxiT^S) C Tx{L ■ x) for generic x E S ( J-transversality), (7.2) 

and is strongly visible if there exists an anti-holomorphic diffeomorphism a 
of D' such that 

a\s = id, (7.3) 
a preserves each L-orbit on D' . (7-4) 

A strongly visible action is visible (j^ Theorem 14]). Furthermore, we 
have: 

Lemma 7.2. Suppose there exists an automorphism a of L such that 

a{g ■ x) = a{g) ■ a{x) {g e L, x E L). (7.5) 
Then, a previsible action satisfying ()7.3|) is strongly visible. 

Proof. Any L-orbit on D' is of the form L ■ x for some x E S. Then, by ()7.3p 
and ()7.5p . we have a{L ■ x) = d-{L) • a{x) = L ■ x. Hence, the condition ()7.4p 
is fulfilled. □ 

Now, let us consider the setting of Theorem 1X1 

Example 7.3. We set 

^{9)-= 9 for g EG = U{n). 

Then a stabilizes subgroups L and H in the setting (jl.lj) in particular, in- 
duces a Lie group automorphism, denoted by the same letter a, of L, and an 
anti-holomorphic diffeomorphism, denoted by a, of the homogeneous space 

G///~i3„„...,„,(n. 
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The L-action on G/H satisfies the compatibihty condition (j7.5|) . Now, we 
consider the totally real submanifold S := Bmi,...,mi (M^) in G/ H. Since cr = id 
on G' = 0{n) and S ~ G' /G' fl H, we have a\s = id. Therefore, if S meets 
every L-orbit on G/H, then the L-action on G/H is not only previsible by 
definition but also is strongly visible by Lemma f7. 21 Hence, the assertion in 
Remark ll. 5. 21 is proved. 

8 Applications to representation theory 

This section gives a fiavor of some applications of Theorem^ to multiplicity- 
free theorems in representation theory. 

In ^U] (see also |l2j and |9ji Theorem 2]), we proved that the multiplicity- 
free property propagates from fibers to spaces of holomorphic sections of 
equivariant holomorphic bundles under a certain geometric condition. The 
key assumption there is strongly visible actions (Definition 17. 1|) on base 
spaces. 

First of all, we observe that one dimensional representations are obviously 
irreducible, and therefore is multiplicity-free. Then, by PU]; this multiplicity- 
free property propagates to the multiplicity-free property of the representa- 
tion on the space 0{G/ H, C\) of holomorphic sections as an L-module for 
any G-equivariant holomorphic line bundle Cx —>■ G/H if {G,L,H) satisfies 
(ii)' (or any of the equivalent conditions) in Theorem 1X1 Then, by a theorem 
of Vinberg-KimelfeldpU] ■ this implies that G/H ~ fi^i, ■■■,m((C") is a spher- 
ical variety of Lq ~ GL{ni, C) x ■ ■ ■ x GL{nk, C), namely, a Borel subgroup 
of Lc has an open orbit on i3mi,...,m;(*^")- 

More generally, applying the propagation theorem of multiplicity-free 
property to higher dimensional fibers, we get from Theorem ^ a new ge- 
ometric proof of a number of multiplicity-free results including: 

• (Tensor product) The tensor product of two irreducible representations 
Tlx and 7r^ of GL{n, C) is multiplicity- free if the highest weight /i G 
is of the form. 



for some a,b & Z {a > b) and some p,q {p + q = n) and if one of the 
following conditions is satisfied: 

1) min(p, q) = 1. 




(a > b). 



(8.1) 
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1) ' a-b=l. 

2) min(p, q) = 2 and A is of the form 

{x, . , X, y, . , y, z, ..^.,z) {x > y > z) (8.2) 

ni 712 "3 

2) ' a - 6 = 2 and A is of the form (Q. 

3) A is of the form ()8.2j) satisfying 

niin(a; -y,y - z, ni, n2, ria) = 1. 

Stembridge jTH] gave a proof of this fact using a combinatorial method 
by a case- by-case argument. He proved also that this exhausts all 
multiplicity-free cases. 

• (Restriction: GL^ I GLp x GLg) The representation tta of GL{n,C) 
is multiplicity- free when restricted to GL{p, C) x GL{q, C) if one of the 
three conditions (1), (2), and (3) is satisfied. 

• (Restriction: GLn I GL^ x GL^j x GLn^) The irreducible repre- 
sentation TT^ of GL{n, C) having highest weight /i of the form (jS.ll) is 
multiphcity-free when restricted to GL{ni, C) x GL{n2, C) x GL{n3, C) 
if min(a — b, p, q) < 2. 

It is noteworthy that the above three multiplicity-free results ( "triunity" ) 
are obtained from a single geometric result, Theorem^ (see [HI Theorems 3.3, 
3.4 and 3.6]). In particular, we get the equivalence (ii) -v^ (iv) in Remark ri.5.31 
by considering the (G x G)-equivariant line bundle C\^C^^G/LxG/H. 

Besides, Theorem 1X1 gives a new geometric proof of yet more multiplicity- 
free theorems (see \2j Theorems 19, 20]): 

• {GLp-GLg duality) The symmetric algebra S{M{p,q;C)) ^ SiCP") is 
multiplicity-free as a representation of GL{p,C) x GL{q,C). 

• (Kac j3]) 5'(CP'') is still multiplicity-free as a representation of GL{p — 
1,C) X GL(g,C). 

• (Panyushev ^H]) Let A/" be a nilpotent orbit of GL{n, C) corresponding 
to a partition (2*'1"'~^p). Then the representation of GL{n,C) on the 
space C[Af] of regular functions is multiplicity-free. 
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All of the examples discussed so far are finite dimensional. As we saw in 
[T^ . we can also expect from strongly visible actions yet more multiplicity- 
free theorems for infinite dimensional representations for both continuous and 
discrete spectra. Applications of Theorem ^ (and its non-compact version) 
to infinite dimensional representations will be discussed in a future paper. 
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